iReasearch学术科研网

摘要：As part of the English-language proficiency requirements for pilots and air traffic controllers, the International Civil Aviation Organization (ICAO) published a rating scale designed to assess pilots' and air traffic controllers' aviation English proficiency. However, it is not clear how this scale was developed. As part of an attempt to address the need for validation, this paper presents a study involving focus group interviews with pilots. Ten pilots listened to performances of test takers taking a variety of aviation English tests. The pilots were asked to rate the acceptability of the pilot's language for (a) communicating with other pilots and (b) radiotelephony communications with air traffic control. The focus groups had two aims: (1) to establish the 'indigenous' assessment criteria pilots use when assessing the language ability of peers and (2) to establish what level is sufficient as the operational level. The results showed that the pilots focused on some but not all of the criteria on the ICAO scale. Whilst listening to the performances, they also often focused on the speakers' technical knowledge. The paper proposes a model of how industry professionals can be involved in the validation of an LSP rating scales. [Copyright The American University; published by Elsevier Ltd.]

摘要：This is a response to the commentary 'English Proficiency Tests and Communication Skills Training for Overseas-Qualified Health Professionals in Australia and New Zealand' by Rosemary Wette, published in Language Assessment Quarterly, Volume 8, Issue 2, 2011. Adapted from the source document

摘要：This study examined the motivation to learn English of Indonesian junior high school pupils, 13-14 years of age, in three distinct contexts: a metropolitan city, a provincial town, and a rural district. Utilizing Dornyei's second language (L2) Motivational Self System as the theoretical framework, this study employed a 50-item questionnaire to sample the views of 527 learners, and a C-test to measure their current proficiency in English. Motivation was found to be similar in strength and character in the two urban settings but significantly different in the rural setting. A positive view of the experience of learning English was the strongest predictor of both motivated learning behavior and L2 proficiency, whereas the Ideal L2 self was only a significant factor among the metropolitan group. Adapted from the source document

摘要：Pseudowords, or non-real words, were introduced to the Yes/No (YN) vocabulary test format to provide a means of checking for overestimation of word knowledge by test takers. The purpose of this study is to assess the assumption that more pseudoword checks (false alarms) indicate more instances of overestimation of word knowledge in YN tests. Thirty English classes in five different Japanese universities with TOEIC(TM) scores ranging from 230 to 730 participated (n = 490). YN test results were compared with a multiple-choice test of the same 96 real words to provide a way to check directly for instances of underestimation and overestimation of word knowledge on the YN tests. Results showed that students from the higher proficiency universities had a slightly higher pseudoword false alarm rate than students from the lower ability universities (4.28% and 3.96%, respectively). However, overestimation rates were considerably lower for these same students from the higher proficiency universities (3.24% and 5.67%, respectively). This discrepancy between false alarm rates and overestimation rates questions the value of pseudowords for measuring overestimation in YN vocabulary tests when student ability levels differ significantly. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This study examined the relationship between scores on the TOEFL Internet-Based Test (TOEFL iBT(TM)) and academic performance in higher education, defined here in terms of grade point average (GPA). The academic records for 2594 undergraduate and graduate students were collected from 10 universities in the United States. The data consisted of students' GPA, detailed course information, and admissions-related test scores including TOEFL iBT, GRE, GMAT, and SAT scores. Correlation-based analyses were conducted for subgroups by academic status and disciplines. Expectancy graphs were also used to complement the correlation-based analyses by presenting the predictive validity in terms of individuals in one of the TOEFL iBT score subgroups belonging to one of the GPA subgroups. The predictive validity expressed in terms of correlation did not appear to be strong. Nevertheless, the general pattern shown in the expectancy graphs indicated that students with higher TOEFL iBT scores tended to earn higher GPAs and that the TOEFL iBT provided information about the future academic performance of non-native English speaking students beyond that provided by other admissions tests. These observations led us to conclude that even a small correlation might indicate a meaningful relationship between TOEFL iBT scores and GPA. Limitations and implications are discussed. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：Performance on integrated tasks requires candidates to engage skills and strategies beyond language proficiency alone, in ways that can be difficult to define and measure for testing purposes. While it has been widely recognized that stimulus materials impact test performance, our understanding of the way in which test takers make use of these materials in their responses, particularly in the context of listening-speaking tasks, remains predominantly intuitive. Recent studies have highlighted the problems associated with content-related aspects of task fulfilment on integrated tasks, but little attempt has been made to operationalize the way in which content from the input material is integrated into speaking performances. Using discourse data from a trial administration of a pilot for an Oxford English language test, this paper investigates how test takers integrate stimulus materials into their speaking performances on an integrated listening-then-speaking summary task, whether these behaviours are reflected in the relevant rating scale and, by implication, whether the test scores assigned according to this scale reflect real differences in the quality of oral performances. An innovative discourse analytic approach was developed to analyse content-related aspects of performance in order to determine if such aspects represent an appropriate measure of the speaking ability construct. Results showed that the measures devised, such as the number of key points included from the input text, and the accuracy with which information was reproduced or reformulated, effectively distinguished participants according to their level of speaking proficiency. The study's findings support the use of this particular task-type and the appropriateness of the associated rating scale as a measure of speaking proficiency, as well as the utility of the devised discourse-based measures for the validation of integrated tasks in other assessment contexts. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This study investigates the effects of topical knowledge on ESL (English as a Second Language) writing performance in the English Language Proficiency Index (LPI), a standardized English proficiency test used by many post-secondary institutions in western Canada. The participants were 50 students with different levels of English proficiency (basic, intermediate, and advanced) attending a Canadian college. Each student wrote two timed-impromptu essays: one responding to a prompt requiring general knowledge about university studies and the other pertaining to specific knowledge about federal politics. Results showed that students across three proficiency levels performed significantly better on the general topic than they did on the specific topic. The specific topic produced lower scores on content due to poor quality and development of ideas, implicit position taking, and a weak conclusion. Students also scored lower on organization and language on the knowledge-specific task because of weaker coherence and cohesion, shorter essays, more language errors, and less frequent use of academic words. Post-test interviews confirmed that participating students were challenged by the prompt that required specific topical knowledge. The study draws attention to the importance of developing appropriate prompts for ESL writing tests. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The content that test-takers attempt to convey is not always included in the construct definition of general English oral proficiency tests, although some English-for-academic-purposes (EAP) speaking tests and most writing tests tend to place great emphasis on the evaluation of the content or ideas in the performance. This study investigated the relative contribution of linguistic criteria and the elaboration of speech content to scores on a test of speaking proficiency. A speaking test was designed and administered to Japanese undergraduates to determine what criteria English teachers associate with general oral proficiency. Nine raters were recruited to rate 30 students' monologues on three topics, using intuitive judgments of oral proficiency (referred to as Overall communicative effectiveness). Following this, they assigned scores to the monologues using five criteria: Grammatical accuracy, Fluency, Vocabulary range, Pronunciation, and Content elaboration/development. The raters were also asked to provide open-ended written comments on the factors contributing to their intuitive judgments. Statistical analyses of the scores -- Rasch measurement, multiple regression, and multivariate generalizability (G) theory analysis -- revealed that Content elaboration/development made a substantive contribution to the intuitive judgments and composite score. The present study enriches our understanding of general oral proficiency and the construct definition of proficiency tests. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：English language development or proficiency (ELD/P) standards promise to play an important role in the instruction and assessment of the language development of English language learner (ELL) pre-K-12 students, but to do so effectively they must convey the progression of student language learning in authentic school contexts for authentic academic purposes. The construct of academic English is defined as the vocabulary, sentence structures, and discourse associated with language used to teach academic content as well as the language used to navigate the school setting more generally. The construct definition is informed by a relatively modest number of empirical studies of textbooks, content assessments, and observations of classroom discourse. The standards of a state with a large ELL population and a large multi-state consortium are then reviewed to illustrate the role of the academic English construct in the standards' coverage of language modalities or domains, levels of attainment or proficiency, grade spans, and the needs of the large number of young English learners. Recommendations and potential strategies for validating, creating, and augmenting standards that reflect authentic uses of academic language in school settings are also made. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：Title III of Public Law 107-110 (No Child Left Behind; NCLB) provided for creation of assessments of English language learners (ELLs) and established, through the Enhanced Assessment Grant program, a platform from which four consortia of states developed ELL tests aligned to rigorous statewide content standards. Those four tests (ACCESS for ELLs, CELLA, ELDA, and MWA) are now in use in one or more states, along with a host of other commercially available or locally developed tests. The tests (those developed by consortia as well as the others) are quite similar in many ways, principally in their contents: Listening, Speaking, Reading, and Writing. Most measure these domains with a combination of multiple-choice (MC) and open-ended (OE) test items. This article provides an overview to the four consortium-developed tests as well as an in-depth analysis of one representative example. It also provides a summary of the characteristics of four commercially available tests. Not surprisingly, the four commercially available tests are rather similar to one another and to the consortium-developed tests in terms of content, psychometric characteristics, and development. The primary difference between the two sets is that the commercially available tests tend to report percentile ranks as well as proficiency levels. Now that the Race to the Top program is in place, we face many of the same challenges we faced a decade ago when NCLB was passed. While the Enhanced Assessment Grant competition emphasized summative assessment, the latest competition emphasizes formative assessment, which gives rise to the hope that educators can not only discover students' strengths and weaknesses with these new tests, but do so in a timely manner and have opportunities to use the information constructively. Current work by at least one organization is encouraging in this regard. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：Assessing Comprehension and Communication in English State-to-State for English Language Learners (ACCESS for ELLS), a large-scale, high-stakes, standards-based, and criterion-referenced language proficiency test developed by the World-Class Instructional Design and Assessment (WIDA) Consortium and administered in US-based K-12 classrooms, is reviewed. Fox notes that ACCESS for ELLS is distinguished from other No Child Left Behind language proficiency tests is (1) the size of the consortium of states using the test, which has grown to 24; (2) the number of resources to situate the test within pedagogical practice; and (3) the care with which WIDA has attempted to systematically align the test with proficiency standards and classroom practice. Adapted from the source document

摘要：One of the mandates of the No Child Left Behind Act is that states show adequate yearly progress in their English language learners' (ELLs) acquisition of English language proficiency. States are required to assess ELLs' English language proficiency annually in four language domains (listening, reading, writing, and speaking) to measure their progress; they are also required to report on a composite comprehension measure. Often the clearest way to effectively monitor students' progress is to measure assessment results across grades on the same scale. In measurement terms, scores from tests across all grade levels can be put on the same scale using vertical scaling. In addition, to help stakeholders understand and interpret the results, these scale scores are often interpreted in terms of proficiency levels. In this article, we use the vertical scaling of WIDA ACCESS for ELLs(TM), a large-scale K-12 Academic English Language Proficiency assessment to illustrate measurement and practical issues involved in this technique. We first give background on the need for vertical scaling. We then assess the literature on vertical scaling and describe the procedures used for WIDA ACCESS for ELLs(TM) to vertically scale test scores and interpret the results in terms of the WIDA ACCESS for ELLs(TM) Proficiency Scale. Next we review several studies that have been conducted to gauge the effectiveness of that scaling. We end the paper with a discussion of the broad issues that arise from vertical scaling. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：With the United States' adoption of a standards-based approach to education, most attention has focused on the large-scale, high-stakes assessments intended to measure students' mastery of standards for accountability purposes. Less attention has been paid to the role of standards-based assessments in the classroom. The purpose of this paper is to discuss key issues and challenges related to the use of standards-based classroom assessments to assess English language learners' English proficiency. First, the paper describes a study of a standards-based classroom assessment of English proficiency in a large urban school district in California. Second, using this study as an example and drawing from the literature in Language Testing on classroom assessment, this paper highlights the major issues and challenges involved in using English proficiency standards as the basis for classroom assessment. Finally, the article outlines a research agenda for the field given current developments in the areas of English proficiency standards and classroom assessment. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The No Child Left Behind (NCLB) Act (US Government, 2001), the current iteration of the Elementary and Secondary Education Act of 1965, makes it clear that states, districts, schools and teachers are accountable for the education of English language learners (ELLs), as well as all other students. To implement an accountability system, NCLB requires states to create content standards (a set of curricular goals and objectives) for English language arts, mathematics, and science. They must also develop and administer assessments that measure student progress toward mastery of these content standards. Finally, states, districts, and schools must demonstrate that they are making adequate yearly progress toward bringing all students to a previously established level of mastery of these standards (called Proficient) with the goal that by the year 2014 all students will achieve this level. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The authors present a model of lexical proficiency based on lexical indices related to vocabulary size, depth of lexical knowledge, and accessibility to core lexical items. The lexical indices used in this study come from the computational tool Coh-Metrix and include word length scores, lexical diversity values, word frequency counts, hypernymy values, polysemy values, semantic co-referentiality, word meaningfulness, word concreteness, word imagability, and word familiarity. Human raters evaluated a corpus of 240 written texts using a standardized rubric of lexical proficiency. To ensure a variety of text levels, the corpus comprised 60 texts each from beginning, intermediate, and advanced second language (L2) adult English learners. The L2 texts were collected longitudinally from 10 English learners. In addition, 60 texts from native English speakers were collected. The holistic scores from the trained human raters were then correlated to a variety of lexical indices. The researchers found that lexical diversity, word hypernymy values and content word frequency explain 44% of the variance of the human evaluations of lexical proficiency in the examined writing samples. The findings represent an important step in the development of a model of lexical proficiency that incorporates both vocabulary size and depth of lexical knowledge features. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：The Test for English Majors (TEM), which aims to measure the English proficiency of Chinese undergraduates, determining whether they have the required language abilities as specified by the National College English Teaching Syllabus for English majors, is highlighted. Adapted from the source document

摘要：Raters are central to writing performance assessment, and rater development -- training, experience, and expertise -- involves a temporal dimension. However, few studies have examined new and experienced raters' rating performance longitudinally over multiple time points. This study uses operational data from the writing section of the MELAB (n = 20,662 ratings), an international exam of English proficiency, to investigate the rating quality of new and experienced raters over three time periods of 12 to 21 months. Rating quality was operationalized in terms of rater severity and consistency, and estimates of those modeled using multi-facet Rasch methodology. Results indicate that, within one particular rating context, (1) novice raters, where initially differing in performance, learn to rate appropriately relatively quickly, (2) raters are able to maintain rating quality over time, and (3) rating volume and rating quality may be related. Implications for rater preparation, rater certification, and the notion of expert rater are discussed. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This paper reports the findings of an empirical study on ESL/EFL teachers' evaluation and interpretation of oral English proficiency as elicited by the national College English Test-Spoken English Test (CET-SET) of China. Informed by debates on the issue of native speaker (NS) norms which have become the focus of attention in recent years, this study addresses the question of whether judgments of language proficiency by non-native English speaking (NNES) teachers, who are currently used to assess performance on the CET-SET, correspond to those of native English speaking (NES) teachers or whether the two groups draw on different constructs of oral proficiency. Data for the study were derived from two sources: unguided holistic ratings given by a group of 19 NES and 20 NNES teachers to CET-SET speech samples from 30 test-takers, and written comments to justify the ratings assigned. Results yielded by both quantitative (MFRM) and qualitative analyses of teacher data, revealed no significant difference in raters' holistic judgments of the speech samples and a broad level of agreement between groups on the construct components of oral English proficiency. However, the analysis of raters' comments revealed both quantitative and qualitative differences in the way NES and NNES teachers weighed various features of the oral proficiency construct in justifying the decisions made. The paper concludes by considering the implications of the study's findings for debates about the native speaker norm as the target for language learners and test-takers. [Reprinted by permission of Sage Publications, Ltd., copyright holder.]

摘要：This nonexperimental study explored the predictive strength of English proficiency levels on academic achievement of middle school students in a sample of 17,470 native English-speaking (NES) students, 558 English language learners (current ELLs), and 500 redesignated fluent English proficient students (former ELLs). Results of multilevel analyses indicated that after controlling for relevant student- and school-level characteristics, former ELLs significantly outperformed current ELL and NES students in reading (effect sizes: 1.07 and 0.52) and mathematics (effect sizes: 0.86 and 0.42). The results support Cummins's (1979, 2000) lower level threshold hypothesis predicting that upon reaching adequate proficiency in the language of schooling and testing, ELLs would no longer experience academic disadvantages. Refinements for the theory and directions for future research are discussed. Adapted from the source document

摘要：This study explored the benefits and drawbacks of synchronous voice-based computer-mediated communication (CMC) in a blended course of English for specific purposes. Quantitative and qualitative data from two groups following the same syllabus, except for the oral component, were compared. Oral tasks were carried out face-to-face with same L1 partners in the control group and through synchronous voice-based CMC with different L1 partners in the experimental group. The analysis included data from general proficiency pre- and post-test scores, oral Power Point presentation grades, students' questionnaires and students' and teachers' diaries. The results showed that achievements were significantly better in the experimental group and that there was also an increase of other positive factors which may effectively contribute both to second language acquisition (SLA) and to solving many of the problems which make speaking skills the weakest skill in foreign language contexts. Adapted from the source document